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1.  Annual  Accomplishments  Summary 

The  objectives  of  this  grant  are  to  develop  new  cooperative  control  protocols  and  trust 
propagation  mechanisms  for  networked  teams  of  nonlinear  dynamical  systems  on 
communication  graphs.  It  is  difficult  to  design  guaranteed-perfonnance  design  control  systems 
for  cooperative  systems  on  graphs  due  to  the  inherent  communication  restrictions  there.  We 
have  developed  a  body  of  knowledge  for  optimal  control  and  adaptive  control  on  graphs  that 
overcomes  these  problems  and  results  in  guaranteed  performance  regardless  of  communication 
topology  constraints.  This  year  we  further  established  numerous  results  concerning  cooperative 
systems  and  multi-player  games.  We  have  developed  reinforcement  learning  adaptive  control 
algorithms  for  solving  optimal  control  and  multi-player  team  games  online  using  data  measured 
along  the  system  trajectories.  This  work  received  the  Best  Paper  Award  at  IJCNN  2010.  We 
have  developed  the  new  concept  of  Dynamic  Graphical  Games.  These  are  games  for  cooperative 
teams  networked  on  communication  graph  topologies,  where  the  standard  definition  of  Nash 
equilibrium  is  of  limited  use.  A  new  concept  of  “Interactive  Nash  Equilibrium”  is  defined. 

In  military  transitions,  we  worked  with  D.  Doman  at  WPAFB  Control  Science  Center  of 
Excellence  where  3  of  our  US  students  were  awarded  summer  fellowships  over  4  years.  A  best 
paper  award  was  received.  We  worked  with  Greg  Hudas  at  US  Army  TARDEC  to  develop  trust 
and  consensus  methods  for  distributed  military  teams.  This  work  received  the  Best  Paper  Award 
in  Autonomous  Systems  at  US  Army  Science  Conference  2010.  In  industry  dual-use  tech 
transfer,  we  work  with  Singapore  Manufacturing  Technology  Institute  in  industrial  machine  tool 
fault  diagnosis.  This  work  received  the  Best  Application  Paper  Award  at  Asian  Control 
Conference  2011. 

This  year  we  published  3  books,  including  Optimal  Control  3ld  edition,  3  journal  special 
issues,  and  11  journal  papers.  Two  of  these  were  cited  as  among  the  mostly  highly  cited 
Automatica  papers.  A  patent  was  submitted  based  on  our  AFOSR  grant  work. 

2.  Objectives  and  Annual  Technical  Accomplishments 

Grant  Objectives 

The  Objectives  of  this  grant  are  to  develop  new  mathematical  frameworks  for  decision-making 
in  autonomous  teams  that  use  local  communication  yet  yield  global  consensus  of  trust,  and  to 
develop  novel  cooperative  motion  control  laws  that  yield  team-wide  desired  behavior.  Military 
battlefield  teams  are  heterogeneous  networks  consisting  of  interacting  humans,  ground  sensors, 
and  unmanned  airborne  or  ground  vehicles.  Given  the  presence  of  enemy  components  and  the 
possibility  of  malicious  attacks  compromising  the  security  of  networked  teams,  a  consensus  must 
be  reached  by  the  team  that  detennines  optimal  team  strategies  and  game  theoretic  solutions 
online  in  real-time  in  changing  environments.  Biological  groups  such  as  flocks,  swarms,  herds 
have  built-in  mechanisms  to  identify  team  members  and  reach  synchronized  motion. 

There  are  two  technical  goals,  and  one  technology  transfer  goal: 

Goal  1:  New  Math  Framework  for  Trust  and  Motion  Consensus  in  Networked  Teams 

Goal  2:  New  Distributed  Feedback  Control  Algorithms  for  Emergent  Team  Behavior 

Tech.  Transfer  Goal:  Work  with  Dr.  David  Doman  at  Dr.  Siva  Banda’s  USAF  Control 

Science  Center  of  Excellence,  WPAFB. 
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A  summary  of  this  year’s  technical  accomplishments  follows. 

Online  Learning  of  Optimal  Cooperative  Strategies  and  Game  Solutions 

In  our  work  with  students  Draguna  Vrabie  and  K.  Vamvoudakis  cited  below  we  have  developed 
new  algorithms  and  theory  for  solving  optimal  control  and  game  theoretic  problems  online  in 
real-time  using  data  measured  along  the  system  trajectories.  We  term  this  Optimal  Adaptive 
Control,  and  the  methods  are  based  on  principles  of  reinforcement  learning.  These  methods 
solve  the  Riccati  equation  or  the  Hamilton  Jacobi  design  equations  online  using  data  measured 
along  the  system  trajectories.  In  the  linear  quadratic  game  case,  for  instance,  the  method  leams 
the  coupled  Riccati  equation  solutions  online  without  ever  explicitly  solving  the  coupled  Riccati 
equation.  In  the  integral  reinforcement  learning  work  of  Vrabie,  the  drift  dynamics  are  not 
needed,  so  the  method  solves  the  Riccati  equation  online  without  knowing  the  system  A  matrix. 

These  optimal  adaptive  learning  techniques  have  been  applied  to  the  control  of  dynamical 
systems  and  also  to  decision-making  in  military  teams.  Optimal  adaptive  control  was  given  for 
the  single-player  optimal  control  problem  for  nonlinear  CT  systems,  which  includes  the  LQR 
case.  The  method  was  given  for  2-player  zero-sum  games,  which  includes  H-infinity  control,  in 
[4]  and  c)  which  won  the  best  paper  award  at  IJCNN  Barcelona  2010.  It  was  extended  to 
multiplayer  nonzero-sum  games  in  [1],  which  was  listed  as  a  top-cited  Automatica  paper.  The 
book  [2]  containing  all  these  results  is  under  contract  to  IET  Press. 

Transition  to  Military  Applications.  With  Greg  Hudas  and  D.  Mikulski  at 
TARDEC/RDECOM,  we  have  also  applied  these  reinforcement  learning  techniques  to  decision 
and  control  in  military  teams.  During  mission  execution  in  military  applications,  the  TRADOC 
Pamphlet  525-66  “Battle  Command  and  Battle  Space  Awareness  Capabilities”  prescribe 
expectations  that  networked  teams  will  perfonn  in  a  reliable  manner  under  changing  mission 
requirements  and  changing  team  and  individual  objectives.  Strategies  for  team  decision  problems 
can  be  formulated  as  optimal  control  problems  or  A-p layer  games  (zero-sum  or  non-zero  sum). 
These  problems  are  normally  solved  off-line  by  solving  associated  matrix  equations  such  as  the 
coupled  Riccati  equations  or  coupled  Hamilton-Jacobi  equations.  However,  using  that  approach, 
players  cannot  change  their  objectives  online  in  real  time  without  calling  for  a  completely  new 
off-line  solution  for  the  new  strategies. 

With  Greg  Hudas  and  D.  Mikulski  at  TARDEC/RDECOM,  we  have  developed  methods 
for  learning  optimal  team  strategies  online  in  real  time  as  team  dynamical  play  unfolds.  See 
work  with  Hudas  and  Mikulski  cited  below.  This  allows  the  objectives  and  costs  of  multi-agent 
teams  to  change  in  real-time  as  the  agents  learn  and  the  game  unfolds.  This  allows  for  truly 
dynamical  team  decisions  where  objective  functions  can  change  in  real  time  and  the  system 
dynamics  can  be  time-varying.  The  paper  b)  won  the  2010  Army  Science  Conference  Best  Paper 
Award  for  Autonomous  Systems.  The  journal  special  issue  [2]  came  out  of  this  work. 

Dynamic  Graphical  Games  for  Networked  Cooperative  Systems. 

We  have  defined  a  new  type  of  differential  game  for  cooperative  networked  systems  on 
communication  graphs  known  as  a  dynamic  graphical  game  [18].  In  graphical  games,  each 
player  interacts  only  with  his  neighbors  in  a  graph  topology.  It  is  shown  that  the  standard  Nash 
definition  is  of  limited  use  for  networked  cooperative  systems  and  a  new  definition  is  given- 
interactive  Global  Nash  Equilibrium.  The  local  concept  of  an  agent’s  best  response  to  its 
neighbors  is  defined.  Methods  are  given  for  solving  graphical  games  online  in  real  time  using 
data  measured  along  the  system  trajectories  as  agents  compete  and  cooperate  during  the  game. 
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This  allows  the  performance  indices  of  the  agents  to  change  in  real-time  as  agents  learn  and  the 
game  unfolds. 

Optimal  and  Adaptive  Control  for  Synchronization  of  Multi-Agent  Systems 

We  have  developed  new  methods  for  control  system  design  for  synchronization  in  distributed 
multi-agent  system.  These  methods  solve  the  well-known  problem  of  interaction  between  the 
controller  design  and  the  graph  topology  constraints.  In  work  with  student  H.  Zhang  below  it  was 
shown  that  optimal  design  of  the  controller  of  each  agent  results  in  guaranteed  synchronization 
on  any  strongly  connected  graph.  In  work  with  H.  Zhang  and  student  Abhijit  Das  a  new  family 
of  adaptive  controllers  was  developed  for  multi-agent  systems  that  synchronizes  cooperative 
systems  with  different  unknown  nonlinear  dynamics.  In  work  with  Gang  Chen  cooperative 
controllers  were  designed  for  Langrangian  cooperative  systems  on  weakly  connected  graph 
topologies.  The  paper  [2]  was  cited  as  the  2nd  top  referenced  paper  in  Automatica  during  July- 
Sept.  2011. 

Transition  to  Military  Applications.  Trust  for  Coalition  Formation  in  Teams 

In  work  with  TARDEC/RDECOM  engineers  Dr.  Greg  Hudas  and  Dariusz  Mikulski,  we  worked 
on  trust  propagation  in  teams  as  a  basis  for  computing  incentives  for  fonnation  of  coalitions. 
Trust  allows  team  players  to  have  confidence  in  other  members  but  also  opens  one  to  a  greater 
possibility  of  harm  if  other  agents  do  not  meet  agreements.  Therefore,  we  have  defined  a  payoff 
for  coalition  formation  called  trust  synergy  and  a  penalty  called  trust  liability.  The  balance 
between  trust  synergy  and  trust  liability  determines  the  evolution  of  coalitions.  Our  paper  b) 
won  the  Best  Paper  Award  in  Autonomous  Systems  at  the  2010  Army  Science  Conference.  Also 
in  this  series  are  the  journal  paper  [10]  and  the  special  issue  [2]. 

3.  Transitions  and  Technology  Transfer 

Transitions  with  WPAFB  Controls  Center  of  Excellence 

In  USAF  transition  we  work  with  Dr.  David  Doman  at  Dr.  Siva  Banda’s  Control  Science  Center 
of  Excellence  at  WPAFB.  Three  of  our  MS  students  have  received  the  USAF  Summer 
Fellowship  to  study  with  Dr.  David  Doman  since  2008.  MS  student  Chris  McMurrough  worked 
with  Dr.  Doman  during  2008  and  2009.  His  project  was  to  design  hardware  boards  for 
interfacing  computers  and  rotorcraft  into  the  USAF  micro-air  vehicle  test  bed.  His  paper  with  D. 
Doman  won  the  Multicore  Graphical  Design  Achievement  Award  at  National  Instruments  Week, 
Austin,  Aug.  2009.  MS  student  Drew  Morgan  received  the  USAF  Summer  Fellowship  to  study 
at  WPAFB  in  2010,  and  Isaac  Weintraub  in  201 1. 

Secondary  Transition  Third  Partner  Relevance 

We  have  strong  connections  with  Dr  Greg  Hudas  at  US  Army  RDECOM  and  are  working  with 
him  on  trust  propagation  and  multi-player  games  for  cooperative  teams  on  networks.  Based  on 
work  described  above,  several  joint  papers  cited  below  have  resulted  and  a  journal  special  issue 
in  J.  Defense  and  Military  Simulation  [2].  RDECOM  PhD  student,  Dariusz  Mikulski,  is  being 
co-advised  by  the  PI.  We  won  the  Best  Paper  Award  for  our  joint  work  b)  on  multi-agent  team 
games  at  Army  Science  Conference,  2010. 
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Dual  Use  Tech  Transfer  to  Industry 

We  work  with  Singapore  Manufacturing  Technology  Institute  SIMTech  to  transition  decision 
and  control  results  from  our  AFOSR  grant  to  industrial  machine  tool  monitoring  and  fault 
diagnosis.  The  cited  papers  of  C.K.  Pang  below  are  the  result.  The  paper  a)  won  the  Best  Paper 
Award  at  the  2011  Asian  Control  Conference.  The  book  [5]  resulted  from  this  collaboration. 

DARPA  Contract  for  Reinforcement  Learning  Control  for  UAV 

We  worked  with  DARPA  through  an  SBIR  I  from  SignalPro,  Inc.,  CEO  Chiman  Kwan,  on 
reinforcement  learning  for  control  of  UAV.  This  transitions  our  AFOSR  results  to  UAV  control. 
We  developed  advanced  nonlinear  controllers  for  a  quadrotor  that  learn  optimal  control  solutions 
through  adaptive  control  online  learning  techniques  based  on  principles  of  reinforcement 
learning. 

4.  New  Discoveries:  Patent  Submitted 

K.  Vamvoudakis,  D.  Vrabie,  F.L.  Lewis,  “Method  for  Learning  Solutions  to  Multi-Player  Games 
in  Real-Time  Using  Integral  Reinforcement  Learning,”  U.S.  Patent  Disclosure,  submitted,  2012. 

5.  Recent  Best  Paper  Awards 

a)  C.K.  Pang,  J.H.  Zhou,  Z.W.  Zhong  and  F.L.  Lewis,  “Industrial  Fault  Detection  and  Isolation  Using 
Dominant  Feature  Identification,”  Proc.  Asian  Control  Conf.,  pp.  1018-1023,  Kaohsiung,  Taiwan,  May 
2011. 

Won  the  Best  Application  Paper  Award  at  Asian  Control  Conference,  Taiwan,  May  2011. 

b)  K.G.  Vamvoudakis,  D.G.  Mikulski,  G.R.  Hildas,  F.L.  Lewis,  and  E.Y.  Gu,  “Distributed  games  for  multi¬ 
agent  systems:  games  on  communication  graphs,"  Army  Science  Conf,  paper  EO-03,  Orlando,  29  Nov-  2 
Dec,  2010. 

Best  Paper  Award  for  Autonomous/Unmanned  Vehicles,  Army  Science  Conf.  2010. 

c)  D.  Vrabie  and  F.L.  Lewis,  “Adaptive  Dynamic  Programming  Algorithm  for  Finding  Online  the  Equilibrium 
Solution  of  the  Two-Player  Zero-Sum  Differential  Game,”  Proc.  Int.  Joint  Conf.  Neural  Networks,  pp.  1-8, 
Barcelona,  July  2010. 

This  paper  won  the  IJCNN  2010  Best  Paper  Award. 

6.  Archival  Publications  during  reporting  period 
Books  Published  or  Accepted  this  year  only 

[1]  F.L.  Lewis,  D.  Vrabie,  and  V.  Syrmos,  Optimal  Control,  third  edition,  John  Wiley  and  Sons,  New  York, 

2012. 

[2]  D.  Vrabie,  K.  Vamvoudakis,  and  F.L.  Lewis,  Optimal  Adaptive  Control  and  Differential  Games  by 
Reinforcement  Learning  Principles,  IET  Press,  to  appear. 

[3]  F.L.  Lewis  and  Derong  Liu,  editors,  Reinforcement  Learning  and  Approximate  Dynamic  Programming  for 
Feedback  Control,  IEEE  Press,  Computational  Intelligence  Series,  to  appear. 

[4]  K.  Sreenath,  M.F.  Mysorewala,  D.O.  Popa,  and  F.L.  Lewis,  Adaptive  Sampling  with  Mobile  WSN: 
Simultaneous  robot  localisation  &  mapping  of parametric  spatio-temporal  fields,  Control  Engineering  Series. 
IET,  2011. 

[5]  C.K.  Pang,  F.L.  Lewis,  T.H.  Lee,  and  Z.Y.  Dong,  Intelligent  Diagnosis  and  Prognosis  of  Industrial 
Networked  Systems,  CRC  Press,  Boca  Raton,  2011. 
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Journal  Special  Issues  Published  or  Accepted  this  year  only 

[1]  S.  Ferrari,  S.  Jagannathan,  and  F.L.  Lewis,  “Special  Issue  on  Approximate  Dynamic  Programming  and 
Reinforcement  Learning,”  Journal  of  Control  Theory  and  Applications,  vol.  9,  no.  3,  Aug.  2011. 

[2]  G.  Hudas,  D.  Mikulski,  and  F.L.  Lewis,  “Special  Issue  on  Intelligent  Behaviors  for  Tactical  Unmanned 
Systems,”  J.  Defense  Modeling  &  Simulation,  vol.  9,  no.  1,  Jan.  2012 

[3]  L.  Zhang,  H.  Gao,  and  F.L.  Lewis,  Special  Section  on  “Advances  in  Theories  and  Industrial  Applications  of 
Networked  Control  Systems,”  IEEE  Transactions  on  Industrial  Informatics,  to  appear  2012. 

Book  Chapters  Published  or  Accepted  this  year  only 

[1]  K.  Vamvoudakis  and  F.L.  Lewis,  “Online  gaming:  real  time  solution  of  nonlinear  two-player  zero-sum 
games  using  synchronous  policy  iteration,”  in  Advances  in  Reinforcement  Learning,  ed.  A.  Mellouk,  InTech 
Publishing,  2011. 

[2]  A.  Das  and  F.L.  Lewis,  “Distributed  adaptive  control  for  networked  multi-robot  systems,”  in  Multi-Robot 
Systems,  Trends  and  Development,  ed.  T.  Yasuda,  InTech  publishing,  Rijecka,  Croatia,  2011. 

[3]  F.L.  Lewis,  “Optimal  Control,”  Chap.  25  in  The  Control  Handbook,  second  edition,  W.S.  Levine  ed.,  CRC 
Press,  2011. 

[4]  D.  Vrabie  and  F.L.  Lewis,  “Approximate  Dynamic  programming,”  Chap.  62  in  The  Control  Handbook, 
second  edition,  W.S.  Levine  ed.,  CRC  Press,  2011. 

[5]  A.  Das,  F.L.  Lewis,  and  K.  Subbarao,  “Sliding  Mode  Approach  to  Control  Quadrotor  Using  Dynamic 
Inversion,"  in  Challenges  and  Paradigms  in  Applied  Robust  Control,  ed.  A.  Batroszewicz,  InTech 
publishing,  Rijecka,  Croatia,  November  2011. 

[6]  K.G.  Vamvoudakis  and  F.L.  Lewis,  “Online  Adaptive  Learning  Solution  of  Multi- Agent  Differential 
Graphical  Games,”  in  "Frontiers  in  Advanced  Control  Systems",  ed.  A.  Lazinica,  Intech  Publishing,  Rijeka, 
Croatia,  to  appear. 

Journal  Papers  Published  or  Accepted  this  year  only 

[1]  K.G.  Vamvoudakis  and  F.L.  Lewis,  “Multi-Player  Non-Zero  Sum  Games:  Online  Adaptive  Learning 
Solution  of  Coupled  Hamilton-Jacobi  Equations,”  Automatica,  vol.  47,  pp.  1556-1569,  2011. 

This  paper  was  listed  as  the  1 7th  most  read  Automatica  article  of  July-Sept.  2011. 

[2]  Shaui  Liu,  L.  Xie,  and  F.L.  Lewis,  “Synchronization  of  multi-agent  systems  with  delayed  control  input 
information  from  neighbors,”  Automatica,  vol.  47,  pp.  2152-2164,  October  2011. 

This  paper  was  listed  as  the  2nd  most  read  Automatica  article  of  July-Sept.  2011. 

[3]  F.L.  Lewis  and  K.G.  Vamvoudakis,  “Reinforcement  learning  for  partially  observable  dynamic  processes: 
adaptive  dynamic  programming  using  measured  output  data,”  IEEE  Trans.  Systems,  Man,  And  Cybernetics- 
Part  B:  vol.  41,  no.  1,  pp.  14-25,  2011. 

[4]  D.  Vrabie  and  F.L.  Lewis,  “Adaptive  dynamic  programming  for  online  solution  of  a  zero-sum  differential 
game,”  J  Control  Theory  App.,  vol.  9,  no.  3,  pp.  353-360,  2011. 

[5]  Gang  Chen  and  F.L.  Lewis,  “Robust  consensus  of  multiple  inertial  agents  with  coupling  delays  and  variable 
topologies,”  Int.  Journal  of  Robust  and  Nonlinear  Control,  vol.  21,  pp.  666-685,  2011. 

[6]  Gang  Chen  and  F.L.  Lewis,  “Leader-following  control  for  multiple  inertial  agents,”  J.  Robust  and  Nonlinear 
Control,  vol.  21,  pp.  925-942,  2011. 

[7]  Gang  Chen  and  F.L.  Lewis,  “Distributed  adaptive  tracking  control  for  synchronization  of  unknown 
networked  Lagrangian  systems,”  IEEE  Trans.  Systems,  Man  and  Cybernetics-  Part  B,  vol.  41,  no.  3,  pp. 
805-816,  June  2011. 
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[8]  Gang  Chen,  F.L.  Lewis,  and  L.  Xie,  “Finite-time  distributed  consensus  via  binary  control  protocols,” 
Automatica,  vol.  47,  pp.  1962-1968,  2011. 

[9]  H.  Zhang,  F.L.  Lewis,  and  Abhijit  Das,  “Optimal  design  for  synchronization  of  cooperative  systems:  state 
feedback,  observer  and  output  feedback,”  IEEE  Trans.  Automatic  Control,  vol.  56,  no.  8,  pp.  1948-1952, 
August  2011. 

[10]  G.  Hudas,  K.G.  Vamvoudakis,  D.  Mikulski,  and  F.L.  Lewis,  "Online  Adaptive  Learning  for  Team  Strategies 
in  Multi-Agent  Systems,"  J.  Defense  Modeling  and  Simulation,”  vol.  9,  no.  1,  pp.  59-69,  Jan.  2012. 

[11]  J.H.  Zhou,  C.K.  Pang,'  F.L.  Lewis,  and  Z.W.  Zhong,  “Dominant  Feature  Identification  for  Industrial  Fault 
Detection  and  Isolation  Applications,”  Expert  Systems  With  Applications,  vol.  38,  no.  8,  pp.  10676-10684, 
Aug.  2011. 

[12]  Gang  Chen  and  F.L.  Lewis,  “Cooperative  controller  design  for  synchronization  of  networked  Lagrangian 
systems,”  J.  Robust  and  Nonlinear  Control,  to  appear,  2012. 

[13]  Gang  Chen  and  F.L.  Lewis,  “Coordination  of  networked  systems  on  digraphs  with  multiple  leaders  via 
pinning  control,”  Int.  J.  Systems  Science,  to  appear,  2012. 

[14]  Abhijit  Das  and  F.L.  Lewis,  “Cooperative  adaptive  control  for  synchronization  of  second-order  systems  with 
unknown  nonlinearities,”  Int.  Journal  of  Robust  and  Nonlinear  Control,  to  appear  2012. 

[15]  H.  Zhang,  F.L.  Lewis,  and  Z.  Qu,  "Lyapunov,  Adaptive,  and  Optimal  Design  Techniques  for  Cooperative 
Systems  on  Directed  Communication  Graphs,"  IEEE  Trans.  Industrial  Electronics,  to  appear,  2012. 

[16]  H.  Zhang  and  F.L.  Lewis,  “Adaptive  cooperative  tracking  control  of  high-order  nonlinear  systems  with 
unknown  dynamics,  Automatica,  to  appear,  2012. 

[17]  H.  Xu,  S.  Jagannathan,  and  F.L.  Lewis,  “Stochastic  Optimal  Control  of  Unknown  Linear  Networked  Control 
System  in  the  Presence  of  Random  Delays  and  Packet  Losses,”  Automatica,  to  appear,  2012. 

[18]  K.G.  Vamvoudakis  and  F.L.  Lewis,  “Multi- Agent  Differential  Graphical  Games:  Policy  Iteration  and  Online 
Adaptive  Learning,”  Automatica,  to  appear,  2012. 

[19]  Guoxiang  Gu,  L.  Maronovici,  and  F.L.  Lewis,  “Consensusability  of  discrete-time  dynamic  multi-agent 
systems,”  IEEE  Trans.  Automatic  Control,  to  appear,  2012. 

[20]  C.K.  Pang,  S.T.  Ng,  F.L.  Lewis,  and  T.H.  Lee,  “Managing  complex  mechatronics  R&D:  a  systems  design 
approach,”  IEEE  Trans.  Systems,  Man,  Cybernetics,  Part  A,  to  appear,  2012. 

[21]  C.  Du,  J.  Mou,  S.  Liu,  J.  Zhang,  and  F.L.  Lewis,  “Structure  Crack  Identification  Based  on  Surface-mounted 
Active  Sensing  Network  With  Time-domain  Feature  Extraction  and  Neural  Network,”  Sensors  & 
Transducers  Journal,  to  appear,  2012. 

Conference  Papers,  Fully  refereed  and  published  in  Proceedings  this  year  only 

[22]  C.K.  Pang,  J.H.  Zhou,  Z.W.  Zhong  and  F.L.  Lewis,  “Industrial  Fault  Detection  and  Isolation  Using 
Dominant  Feature  Identification,”  Proc.  Asian  Control  Conf.,  pp.  1018-1023,  Kaohsiung,  Taiwan,  May 
2011. 

Won  the  Best  Application  Paper  Award  at  Asian  Control  Conference,  Taiwan,  May  2011. 

[23]  Z.Qu,  C.  Li,  F.L.  Lewis,  “Cooperative  control  based  on  distributed  estimation  of  network  connectivity,” 
Proc.  American  Control  Conference,  pp.  3441-3446,  San  Francisco,  2011. 

[24]  G.  Chen  and  F.L.  Lewis,  “Synchronizing  networked  Lagrangian  systems  via  binary  control  protocols,”  Proc. 
IF  AC  World  Congress,  Milan,  Italy,  Aug.  2011. 

[25]  A.  Gasparri,  D.  Di  Paola,  G.  Ulivi,  D.  Naso,  F.L.  Lewis,  “Decentralized  task  sequencing  and  multiple 
mission  control  for  heterogeneous  robotic  networks,”  Proc.  Int.  Conf.  Robotics  and  Automation,  Shanghai, 
May  2011. 

[26]  G.  Gu,  L.  Marinovici,  and  F.L.  Lewis,  “Consensusability  of  discrete-time  multi-agent  systems  under  state 
feedback  control,”  Proc.  Chinese  Control  Conf.,  Yantai,  March  2011. 
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[27]  D.G.  Mikulski,  F.L.  Lewis,  E.Y.  Gu,  and  G.R.  Hildas,  “Trust  dynamics  in  multi-agent  coalition  formation,” 
Proc.  SPIE  Defense  Symposium,  Orlando,  April  2011. 

[28]  Z.  Qu,  C.  Li,  and  F.L.  Lewis,  "Cooperative  Control  Based  on  Distributed  Estimation  of  Network 
Connectivity,”  Proc.  American  Control  Conf.,  San  Francisco,  June  2011. 

[29]  K.G.  Vamvoudakis  and  F.L.  Lewis,  “Multi-agent  differential  graphical  games,”  Proc.  Chinese  Control 
Conf.,  pp.  4932-4939,  Yantai,  China,  July,  2011. 

[30]  K.G.  Vamvoudakis  and  F.L.  Lewis,  “Policy  iteration  algorithm  for  distributed  networks  and  graphical 
games,”  Proc.  IEEE  Conf  Decision  &  Control,  pp.  128-135,  Orlando,  Fla,  Dec.  2011. 

[31]  K.  Vamvoudakis,  D.  Vrabie,  and  F.L.  Lewis,  “Online  Adaptive  Learning  of  Optimal  Control  Solutions 
Using  Integral  Reinforcement  Learning,”  Proc.  IEEE  Symp.  ADPRL,  Paris,  11-15  April  2011. 

[32]  E.  Stingu  and  F.L.  Lewis,  “An  Approximate  Dynamic  Programming  Based  Controller  Implementation  for 
an  Underactuated  6DoF  Quadrotor,”  Proc.  IEEE  Symp.  ADPRL,  11-15  Paris,  April  2011. 

[33]  K.  Vamvoudakis  and  F.L.  Lewis,  “Online  Learning  Solution  of  Coupled  Hamilton- Jacobi  and  Coupled 
Riccati  Equations,”  Proc.  IEEE  Multi-conference  on  Systems  and  Control,  Denver,  CO,  Sept.  2011. 

[34]  S.G.  Khan,  G.  Herrmann,  F.L.  Lewis,  T.  Pipe,  and  C.  Melhuish,  “A  novel  Q-learning  based  Cartesian  model 
reference  compliance  controller  implementation  for  a  humanoid  robotic  arm,”  Proc.  IEEE  Int.  Conf. 
Robotics,  Automation  and  Mechatronics,  CIS-RAM  2011,  paper  24,  Qingdao,  China,  Sept.,  2011. 

[35]  K.Vamvoudakis  and  F.L.  Lewis,  “Policy  Iteration  Algorithm  for  Distributed  Networks  and  Graphical 
Games,”  Proc.  IEEE  Conf.  Decision  and  Control,  Orlando,  Dec.  2011. 

[36]  C.K.  Pang,  T.S.  Ng,  F.L.  Lewis,  T.H.  Lee,  “A  systems  design  approach  to  manage  mechatronics  R&D,” 
Proc.  IEEE  Int.  Conf.  Robotics,  Automation  and  Mechatronics,  CIS-RAM  2011,  pp.  136-141,  Qingdao, 
China,  Sept.,  2011. 

[37]  C.K.  Pang,  C.V.  Le,  O.P.  Gan,  X.M.  Chee,  M.  Luo,  C.H.  Leng,  F.L.  Lewis,  “Intelligent  energy  audit  and 
machine  management  for  energy-efficient  manufacturing,”  Proc.  IEEE  Int.  Conf.  Robotics,  Automation  and 
Mechatronics,  CIS-RAM  201 1,  pp.  142-147,  Qingdao,  China,  Sept.,  2011. 

[38]  C.V.  Le,  C.K.  Pang,  F.L.  Lewis,  O.P.  Gan,  H.L.  Chan,  “Intelligent  Dynamic  Resource  Assignment  for 
Energy-Efficiency  in  Industrial  Stamping  Machines,”  Proc.  IEEE  Industrial  Electronics  Society  (IECON 
2011),  pp.  4131-4136,  Melbourne,  Nov.  2011. 

7.  Awards 

PI  Awards  This  Year 

Visiting  Scholar  Fellow,  Singapore  Institute  of  Manufacturing  Technology,  SIMTech,  A-Star, 

2011. 

Four  International  Invited  Plenary  Talks  in  16  months. 

Student  Awards  Throughout  Grant  Duration 

Chris  McMurrough,  MS  student.  “Micro  Flapping  Air  Vehicle.”  Received  the  Air  Force 

Summer  Fellowship  to  work  at  Wright  Patterson  AFB  with  Dr.  David  Doman  and  Dr.  Siva 

Banda,  in  2009. 

UTA  Honors  College  Fellowship,  2008,  2009. 

C.  McMurrough,  K.  French,  D.  Doman,  "Real-Time  MAV  Flight  Control  System  Testbed” 

This  paper  won  the  Multicore  Graphical  Design  Achievement  Award  at  National 
Instruments  Week,  Austin,  Aug.  2009 
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Drew  Morgan.  MS  student.  “Minirobot  Designs  for  Swarm  Motions.”  Received  the  Air  Force 
Summer  Fellowship  to  work  at  Wright  Patterson  AFB  with  Dr.  David  Doman  and  Dr.  Siva 
Banda,  in  2010. 

Isaac  Weintraub.  MS  student.  “Micro  Flapping  Air  Vehicle.”  Received  the  Air  Force  Summer 
Fellowship  to  work  at  Wright  Patterson  AFB  with  Dr.  David  Doman  and  Dr.  Siva  Banda,  in 
2011. 

8.  Students  Supported 
PhD  students  graduated  this  year 

1.  K.  Vamvoudakis,  Online  Learning  Algorithms  For  Differential  Dynamic  Games  And 

Optimal  Control,  May  2011. 

Best  Paper  Award  for  Autonomous/Unmanned  Vehicles,  Army  Science  Conf  Orlando,  29 
Nov-  2  Dec,  2010. 

2.  E.  Stingu,  Intelligent  Control  and  Cooperation  for  Mobile  Robots,  December  2011. 

Students  Supported  During  Duration  of  Grant 

(AFOSR  and  leveraging  funds) 

Draguna  Vrabie,  female  student,  received  PhD  at  UTA  in  December  2009. 

Senior  Research  Scientist,  United  Technologies  Research  Center,  East  Hartford,  CT 

Abhijit  Das,  received  PhD  at  UTA  in  August  2010. 

Now  at  Caterpillar,  Inc. 

Kyriakos  Vamvoudakis,  received  PhD  at  UTA  in  May  2011. 

Center  for  Control,  Dynamical-Systems,  and  Computation  (CCDC),  UC  Santa  Barbara 

E.  Stingu,  received  PhD  at  UTA  in  December  2011. 

Now  at  UAV  Design 
Kristian  Movric,  PhD  student  at  UTA. 

M.  Aurangzeb,  PhD  student  at  UTA. 

M.  Abouheaf,  PhD  student  at  UTA. 

US  Citizen  Students:  All  have  worked  with  Dr  Siva  Banda’s  group  at  WPAFB. 

Chris  McMurrough,  received  MS  Degree  at  UTA  May  2010. 

Drew  Morgan,  received  MS  degree  at  UTA  in  May  2011. 

Isaac  Weintraub,  MS  student  at  UTA. 
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